feat(benchmark): add append/pk table benchmark by lucasfang · Pull Request #302 · alibaba/paimon-cpp

lucasfang · 2026-05-26T01:56:12Z

Purpose

Linked issue: close #100

Add a Google Benchmark based performance suite covering:
1. BM_Write (append table write)
2. BM_Read (append table read with prefetch_parallel variants)
3. BM_PK_Write (primary key table write)
4. BM_MOR_Read (primary key MOR read with prefetch_parallel variants)
Add a benchmark entrypoint and custom CLI parsing for:
1. --paimon_source_parquet
2. --paimon_external_table_path
3. --paimon_file_format
4. --paimon_pk_columns
5. --paimon_option (repeatable)
Add benchmark helper utilities for:
1. Validation and skip behavior for invalid configurations
2. Shared read iteration execution
3. External table read mode (skip pre-write stage)
4. Source parquet loading and cache reuse
Build integration updates:
1. Add PAIMON_BUILD_BENCHMARKS option (default OFF)
2. Add benchmark subdirectory and benchmark target
3. Add add_paimon_benchmark macro and benchmark labels
4. Add benchmark dependency discovery via FindbenchmarkAlt
5. Add PAIMON_BENCHMARK_BUILD_VERSION=1.9.1 in third_party versions

Tests

UT:

CliOptionParsingTest.ConsumeCliOptionWorks
CliOptionParsingTest.ParseCsvColumnsWorks
CliOptionParsingTest.ParseCsvColumnsRejectsInvalidInput
CliOptionParsingTest.ParseDelimitedOptionsWorks
CliOptionParsingTest.ParseDelimitedOptionsRejectsInvalidInput
CliOptionParsingTest.ParseStringOptionArgWorksForEqualsAndSeparatedForms
CliOptionParsingTest.ParseStringOptionArgRejectsMissingValue
CliOptionParsingTest.ParseCsvOptionArgAndDelimitedRepeatableOptionArgWorks

Benchmark smoke:

BM_Write
BM_Read/1, BM_Read/2, BM_Read/4
BM_PK_Write
BM_MOR_Read/1, BM_MOR_Read/2, BM_MOR_Read/4

IT:

No new integration tests in this PR (benchmark and build integration scope)

API and Format

No change to public API under include.
No storage format or protocol compatibility changes.
Changes are limited to benchmark executable behavior and build wiring.

Documentation

No user-facing documentation change required for this PR.
Benchmark usage documentation can be added separately if needed.

Generative AI tooling

Generated-by: GitHub Copilot (GPT-5.3-Codex)

Copilot

Pull request overview

This PR introduces an optional Google Benchmark-based performance suite for Paimon (append-table write/read and PK-table write/MOR read), along with CMake/third-party wiring to build and run the benchmarks via CTest labels.

Changes:

Add a PAIMON_BUILD_BENCHMARKS build option and a benchmark CTest label/target integration.
Vendor/resolve Google Benchmark as a dependency (bundled/system) and add a FindbenchmarkAlt.cmake module.
Add a benchmark executable with custom CLI parsing and shared helper utilities + unit tests for CLI parsing.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 9 comments.

Show a summary per file

File	Description
third_party/versions.txt	Adds Google Benchmark version metadata for bundled dependency download.
CMakeLists.txt	Adds `PAIMON_BUILD_BENCHMARKS` option + benchmark CTest target/labels and subdir wiring.
cmake_modules/ThirdpartyToolchain.cmake	Adds Benchmark dependency resolution and bundled build rule.
cmake_modules/FindbenchmarkAlt.cmake	Adds a “system” find module for Google Benchmark.
cmake_modules/DefineOptions.cmake	Adds `PAIMON_BUILD_BENCHMARKS` and `Benchmark_SOURCE` options.
cmake_modules/BuildUtils.cmake	Adds `add_paimon_benchmark` / `add_benchmark_case` helpers and CTest labeling.
benchmark/CMakeLists.txt	Defines benchmark executable target and CLI parsing unit test target.
benchmark/read_write_benchmark.cpp	Benchmark entrypoint with custom CLI parsing + Google Benchmark init/run.
benchmark/cli_option_parsing.h	Inline parsing helpers for custom benchmark CLI options.
benchmark/cli_option_parsing_test.cpp	GTest coverage for CLI parsing helpers.
benchmark/benchmark_suite.h	Declares benchmark runner functions and CLI helpers.
benchmark/benchmark_suite.cpp	Implements benchmark suite: table setup, write/commit, read iterations, caching, CLI handling.
benchmark/benchmark_helpers.h	Declares shared validation/skip and read-iteration helpers.
benchmark/benchmark_helpers.cpp	Implements validation/skip behavior and shared read-iteration runner.
benchmark/benchmark_case_write.cpp	Registers BM_Write benchmark.
benchmark/benchmark_case_read.cpp	Registers BM_Read benchmark variants.
benchmark/benchmark_case_pk_write.cpp	Registers BM_PK_Write benchmark.
benchmark/benchmark_case_mor_read.cpp	Registers BM_MOR_Read benchmark variants.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

zjw1111 · 2026-05-28T10:33:59Z

I found two issues in the current PR head:

[P1] benchmark/CMakeLists.txt unconditionally links GNU ld flags and rt (-Wl,--whole-archive, -Wl,--no-as-needed, -Wl,--as-needed, rt). This makes PAIMON_BUILD_BENCHMARKS=ON fail on non-Linux/ELF toolchains such as macOS, where those linker flags and librt are not available. Please guard these flags/libraries by platform or linker capability.
[P2] The PK/MOR benchmarks create tables with bucket=4, but MakeRecordBatch() always calls SetBucket(0), so all benchmark data is written into bucket 0. FileStoreWrite only validates that the bucket is in range; it does not re-hash rows by primary key. As a result, the PK/MOR benchmark layout does not represent a normal multi-bucket PK table. Please either distribute rows to buckets by PK hash or make the PK benchmarks explicitly use bucket=1.

The earlier checksum, URL_HASH, temp-directory cleanup, and help-exit concerns appear to be fixed in the current head.

Copilot AI review requested due to automatic review settings May 26, 2026 01:56

Copilot started reviewing on behalf of lucasfang May 26, 2026 01:56 View session

Copilot AI reviewed May 26, 2026

View reviewed changes

zjw1111 reviewed May 28, 2026

View reviewed changes

Comment thread benchmark/CMakeLists.txt Outdated

Comment thread benchmark/CMakeLists.txt Outdated

Comment thread cmake_modules/ThirdpartyToolchain.cmake Outdated